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ABSTRACT 

This paper presents novel adaptive reduced-rank filtering algorithms 
based on joint iterative optimization of adaptive filters. The novel 
scheme consists of a joint iterative optimization of a bank of full- 
rank adaptive filters that constitute the projection matrix and an adap- 
tive reduced-rank filter that operates at the output of the bank of 
filters. We describe least squares (LS) expressions for the design 
of the projection matrix and the reduced-rank filter and recursive 
least squares (RLS) adaptive algorithms for its computationally ef- 
ficient implementation. Simulations for a space-time interference 
suppression in a CDMA system application show that the proposed 
scheme outperforms in convergence and tracking the state-of-the-art 
reduced-rank schemes at about the same complexity. 

Index Terms — Adaptive filters, iterative methods, RLS algo- 
rithms, space-time processing. 



1. INTRODUCTION 

In adaptive filtering 1 1 1, there is a huge number of algorithms with 
different trade-offs between performance and complexity. Among 
them, recursive least squares (RLS) algorithms arise as the preferred 
choice with respect to convergence performance. A challenging prob- 
lem which remains unsolved by conventional techniques is that when 
the number of elements in the filter is large, the algorithm requires a 
large number of samples to reach its steady-state behavior In these 
situations, even RLS algorithms require an amount of data propor- 
tional to 271/ |1| in stationary environments, where M is the fil- 
ter length, to converge and this may lead to unacceptable perfor- 
mance. Reduced-rank filtering |2|-|10| is a powerful and effective 
technique in low sample support situations and in problems with 
large filters. The advantages of reduced-rank adaptive filters are their 
faster convergence speed and better tracking performance than exist- 
ing techniques when dealing with large number of weights. Further- 
more, in dynamic scenarios large filters usually fail or provide poor 
performance in tracking signals embedded in interference. Several 
reduced-rank methods and systems have been proposed in the last 
several years, namely, eigen-decomposition techniques ||3l-|4), the 
multistage Wiener filter (MWF) [6, 7| and the auxiliary vector fil- 
tering (AVF) algorithm |9]. The main problem with the best known 
techniques is their high complexity and the fact that there is no joint 
optimization of the mapping that carries out dimensionality reduc- 
tion and the reduced-rank filter 

In this work we propose an adaptive reduced-rank filtering scheme 
based on combinations of adaptive filters with RLS algorithms. The 
novel scheme consists of a joint iterative optimization of a bank 
of full-rank adaptive filters which constitutes the projection matrix 
and an adaptive reduced-rank filter that operates at the output of 
the bank of full-rank filters. The essence of the proposed approach 
is to change the role of adaptive filters. The bank of adaptive fil- 
ters is responsible for performing dimensionality reduction, whereas 
the reduced-rank filter effectively estimates the desired signal. De- 



spite the large dimensionality of the projection matrix and its as- 
sociated slow learning behavior, the proposed and existing fTl, ^ 
reduced-rank techniques enjoy in practice a very fast convergence. 
The reason is that even an inaccurate or rough estimation of the 
projection matrix is able to provide an appropriate dimensionality 
reduction for the reduced-rank filter, whose behavior will govern 
most of the performance of the overall scheme. We describe least 
squares (LS) expressions for the design of the projection matrix and 
the reduced-rank filter along with RLS adaptive algorithms for its 
computationally efficient implementation. The performance of the 
proposed scheme is assessed via simulations for a space-time inter- 
ference suppression application in DS-CDMA systems. 

This work is organized as follows. Section 2 states the reduced- 
rank estimation problem. Section 3 presents the novel reduced-rank 
scheme, the joint iterative optimization and the LS design of the fil- 
ters. Section 4 derives RLS algorithms for implementing the pro- 
posed scheme. Section 5 shows and discusses the simulations, while 
Section 6 gives the conclusions. 



2. REDUCED-RANK LEAST SQUARES PARAMETER 
ESTIMATION AND PROBLEM STATEMENT 

The exponentially weighted LS estimator is the parameter vector 
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following cost function 



which is designed to minimize the 
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j]"^ is the in- 



where d[l] is the desired signal, r[i] = [r^ 

put data, (■)^ and (•)^ denote transpose and Hermitian transpose, 
respectively, and A stands for the forgetting factor. The set of pa- 
rameters w[j] can be estimated via standard stochastic gradient or 
LS estimation techniques |1|. However, the laws that govern the 
convergence behavior of these estimation techniques imply that the 
convergence speed of these algorithms is proportional to M, the 
number of elements in the estimator. Thus, large M implies slow 
convergence. A reduced-rank algorithm attempts to circumvent this 
limitation in terms of speed of convergence by reducing the num- 
ber of adaptive coefficients and extracting the most important fea- 
tures of the processed data. This dimensionality reduction is accom- 
plished by projecting the received vectors onto a lower dimensional 
subspace. Specifically, consider an AI x D projection matrix S_d [i] 
which carries out a dimensionality reduction on the received data as 
given by 

f[J]=SgWrW (2) 

where, in what follows, all D-dimensional quantities are denoted 
with a "bar". The resulting projected received vector f[i] is the in- 
put to a tapped-delay line filter represented by the D vector w[i] — 
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for time interval i. The estimator output corre- 



spending to the ith time instant is 



(3) 



If we consider the LS design in (1) with the reduced-rank parameters 
we obtain 



w[i]=R-^[i]pH 



(4) 



whereR[i] = J2Ui y'^r[l]r"[l] = Sg[i]R[i]Si3[i] is the reduced- 



is the full-rank 



Sg[i]p[i] is the 



rank covariance matrix, R[i] — X]I=i ^^ i"[^]i" 
covariance matrix, p[i] — X^I^i '^*~''^*W*'W 
cross-correlation vector of the reduced-rank model and the vector 
p[i] = ^l^iy~''d''[l]r[l]] is the cross-correlation vector of the 
full-rank model. The associated sum of error squares (SES) for a 
rank D estimator is expressed by 



SES 
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where ctJ — X^I^i A*~'|d(i)p. Based upon the problem statement 
above, the rationale for reduced-rank schemes can be simply put as 
follows. How to efficiently (or optimally) design a transformation 
matrix Sd [i] with dimension M x D that projects the observed data 
vector r[i] with dimension M x 1 onto a reduced-rank data vector 
f[i] with dimension D x 1? In the next section we present the pro- 
posed reduced-rank approach. 

3. PROPOSED REDUCED-RANK SCHEME AND LEAST 
SQUARES DESIGN 

In this section we detail the principles of the proposed reduced-rank 
scheme using a projection operator based on adaptive filters and 
present a least squares (LS) design approach for the estimators. The 
novel scheme, depicted in Fig. 1, employs a projection matrix S_d [i] 
with dimension M x D, that is responsible for the dimensionality 
reduction, to process a data vector r[i] with dimension M x 1 and 
map it into a reduced-rank data vector f[i]. The reduced-rank filter 
w[i] with dimension D x 1 processes the reduced-rank data vector 
f[i] in order to yield a scalar estimate x[i]. The projection matrix 
SdIi] and the reduced-rank filter \v[i] are jointly optimized in the 
proposed scheme according to the LS criterion. 



that provides a gain. For D > 1, the signal processing tasks are 
changed and the full-rank filters compute a subspace projection and 
the reduced-rank filter estimates the desired signal. 

We describe LS expressions for the design of the projection ma- 
trix and the reduced-rank filter along with RLS adaptive algorithms 
for its computationally efficient implementation. Let us consider the 
exponentially-weighted LS expressions for the filters S_d [i] and w[i] 
can be computed via the cost function given by 
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By minimizing (7) with respect to w [i] , the reduced-rank filter weight 
vector becomes 

w[i\=K-^[{\p\i] (8) 

where pW = SE[^j:]^,X^-'d*[l]r[l] = j:U y-'d'[l]f[% 

R[i] = Sg[i]5:;;^-^A'-'r[Z]r*f[Z]SD[i]. By minimizing (7) with 
respect to Sd [i] we obtain 



SD[i\='R.-'[{\PD\i]R-^[i 



(9) 



where Pofi] = X];=i -^' 'd*Wr[Z]w^[i], the covariance matrix is 
R-W = ELi^'"'r[Z]r«[Z] andR»[i] = ELi ^'''w[Z]w^[Z]. 
The associated SES is 
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where ctJ — E;=i A'~'|d[Z]p. Note that the expressions in (8) and 
(9) are not closed-form solutions for \v[i] and S_D[i] since (8) is a 
function of Soli] and (9) depends on w[i] and thus they have to be 
iterated with an initial guess to obtain a solution. The key strategy 
lies in the joint optimization of the filters. The rank D must be set by 
the designer to ensure appropriate performance. The expressions in 
(8) and (9) require the inversion of matrices. In order to reduce the 
complexity, we employ the matrix inversion lemma and derive RLS 
algorithms in the next section. The rank D must be set by the de- 
signer to ensure appropriate performance and the reader is refeiTed 
to (I5( for rank selection methods. In the next section, we seek iter- 
ative solutions via adaptive algorithms. 

4. PROPOSED RLS ALGORITHMS 
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Fig. 1. Proposed Reduced-Rank Scheme. 
Specifically, the projection matrix is structured as a bank of D 
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D, with 
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The 



full-rank filters Sd[i] 

dimensions M x 1 as given by S_d [i] — [ Sj'' | s 
output estimate x[i] of the reduced-rank scheme as a function of the 
received data r[i], the projection matrix Sd [i] and the reduced-rank 
filter w[i] is 



x[i] = w [iJSd [J]r[J] = vv [i]r[i 



(6) 



Note that for D = 1, the novel scheme becomes a conventional 
full-rank filtering scheme with an addition weight parameter wd 



In this section we propose RLS algorithms for efficiently implement- 
ing the LS design of the previous section. Firstly, let us consider the 
expression in (8) with its associated quantities, i.e. the matrix R[i] — 

ELi A*-'f[Z]f^[Z] and the vector p[i\ = J^j^j X'-'d'[l]r[l], de- 
fine ^[i] = R^""" [i] and rewrite p[i] as p[i] — \p[i — 1] +d* [i]f[i]. 
We can write (8) in an alternative form as follows 

w[i] = #[i]p[i] = A*[i]p[i - 1] + #[i]f [i]d*[i] 

= #[i - l]p[i - 1] -k[i]f^[i]#[i - l]p[i - 1] + #[i]f[i]d*[i 



\v[i - 1] - k\i]f"[i]w[i - 1] + k[i]d*[i] 
w[i - 1] + k[i\ [d*[i] - f^[z]w[i - 1]] 



(11) 



By defining ^[i] — d[i] — w^[j — l]r^[i] we arrive at the proposed 
RLS algorithm for estimating w[i] 

w[i]^w[i-l] + -k[i\C[i\ (12) 

where the so-called Kalman gain vector is given by 

\-^^[i-l]f[i\ 



kfil = 



l + A-if»[i]*[i- l]r[i\ 



(13) 



and the update for the matrix inverse ^[i] employs the matrix inver- 
sion lemma 11] 



*[i] = A~^*[i - 1] - A'^k^r [j]*[j - 1] 



(14) 



Note that the proposed RLS algorithm given in (12)-(14) is simi- 
lar to the conventional RLS algorithm 11], except that it works in 
a reduced-rank model with a _D x 1 input f[i] = S|5[i]r[i], where 
the M X D matrix Sd is the projection matrix responsible for di- 
mensionality reduction. Now let us present the second part of the 
proposed RLS algorithms, in which we detail the design of Sufi]. 
Let us define P[j] = R"^[i], Qw[i - 1] = P^M«]- Po[i] = 
XPoli — 1] -I- d*[i]r[i]w^[j] and rewrite the expression in (9) as 
follows 

Soli] = RWPl3[j]Pw[J - 1] = P[i]PD WQw[i - 1] 

= XF\i]FD[i - l]Qw[i - 1] +d*[J]P[J]r[j]w^[i]Qw[i] 
= Si5[i - 1] - k\i]F[i - l]Pi3[i - l]Q^[i] 

+ d''[i\P[i\r[i\w"\i]Q^[i\ 

= Sl5[i - 1] - k[J]P[J - 1]P D[i - l]Qw[i] 

+ d*[i]k\i]w"\i]Q^\i] 

(15) 

By defining the vector t[i] = Qw[J]w[i] and using the fact that 

f"[i -l]=r"[i- 1]S D[i - 1] we arrive at 

Sd [i] ^Sd[i-1]+ kH (d* [i\t" [i\ - f" [i\) (16) 

where the Kalman gain vector for the estimation of Sn [i] is 

A-^P[i - l]r[i\ 



kM = 



l + A-ir«[i]P[i-l]r[i] 



(17) 



and the update for the matrix P[i] employs the matrix inversion 
lemma |T| 

P[i] = A"^P[i - 1] - A"^k[i]r"[i]P[i - 1] (18) 

the vector t[i] is updated as follows 

A"^Qw[i- l]w[i- 1] 



tm = 



1 + A-iw»[i - l]Qw[i - llw[i - 1] 



(19) 



and the matrix inversion lemma is used to update Qw [i] as described 
by 

QwW = A"^Qw[i - 1] - \'^t[i\w"[i - 1] (20) 

The equations (16)-(20) constitute the second part of the proposed 
RLS algorithms and are responsible for estimating the projection 
matrix Si5[i]. The computational complexity of the proposed RLS 
algorithms is 0{D^) for the estimation of w[i] and 0{M^) for the 
estimation of Si5[i]. Because D « M, as will be explained in 
the next section, the overall complexity is in the same order of the 
conventional full-rank RLS algorithm (0{AP)) |T]. 

5. SIMULATIONS 

The performance of the proposed scheme is assessed via simula- 
tions for space-time CDMA interference suppression. We consider 
the uplink of DS-CDMA system with symbol interval T, chip period 
Tc, spreading gain A'^ — T/Tc, K users and equipped with J ele- 
ments in a uniform antenna array. The spacing between the antenna 
elements is d = Ac/2, where Ac is carrier wavelength. Assuming 
that the channel is constant during each symbol and the base station 



receiver is synchronized with the main path, the received signal af- 
ter filtering by a chip-pulse matched filter and sampled at chip rate 
yields the JM x 1 received vector 



v[i] =^Akbk[i - l]pfc[i - V\+Akbk[i\pk[i 
*,— 1 
+ A-fefe[i + l]pfe[i + l] + n[i], 



(21) 



where M = N + Lp — 1, the complex Gaussian noise vector is 
n[j] = [ni[i] ... njM[i]Y with £[n[i]n^[i]] = a^I, {Y and 
(■)^ denote transpose and Hermitian transpose, respectively, and 
£[■] stands for expected value. The spatial signatures are f>k[i — 

1] = TkUk[i - 1], PfcW = TkHk[i] and f>k[i] = TkHk[i + 
1], where jFfc, J^k. and fik are block diagonal matrices with one- 
chip shifted versions of segments of the signature sequence Sfe = 
[afc(l) . . . aki^N)^ of user k. The JLp x 1 space-time channel vec- 
tor is given by Ukli] = [^lAi]\ ^l.i[i 



•AD 



\Kj- 



\i\ 1 ^ with 



] being the channel of user k at an- 
tenna element / with their associated DoAs (f>k,m- 

For all simulations, we use the initial values w[0] = [10 ... 0]"^ 
and Si5[0] = [lo Od,jm-d^ , assume L = 9 as an upper bound, 
use 3-path channels with relative powers given by 0, —3 and —6 dB, 
where in each run the spacing between paths is obtained from a dis- 
crete uniform random variable between 1 and 2 chips and average 
the experiments over 200 runs. The DOAs of the interferers are uni- 
formly distributed in (0, 27r/3). The system has a power distribution 
among the users for each run that follows a log-normal distribution 
with associated standard deviation equal to 1.5 dB. We compare the 
proposed scheme with the Full-rank 1 1 1, the MWF |7 1 and the AVF 
J9j techniques for the design of linear receivers, where the reduced- 
rank filter w[i] with D coefficients provides an estimate of the de- 
sired symbol for the desired used (user 1 in all experiments) using the 
bit error rate (BER) [7 1. We consider the BER performance versus 
the rank D with optimized parameters (forgetting factors A = 0.998) 
for all schemes. The results in Fig. 2 indicate that the best rank for 
the proposed scheme is D = 4 (which will be used in the remaining 
experiments) and it is very close to the optimal MMSE. Studies with 
systems with different processing gains show that D is invariant to 
the system size, which brings considerable computational savings. 



N=1 6, K=1 users, E /N =7 dB, f T=0.00005, Data support: 500 symbols 
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Fig. 2. BER performance versus rank (D). 



We compare the proposed scheme with the Full-rank yj, the 



MWF {T\ and the AVF f9l techniques for the design of Mnear re- 
ceivers, where the reduced-rank filter w[i] with D coefficients pro- 
vides an estimate of the desired symbol for the desired used (user 
1 in all experiments) using the signal-to-interference-plus-noise ra- 
tio (SINR) |7|. We consider the BER performance versus the rank 
D with optimized parameters (forgetting factors A) for all schemes. 
The results in Fig. 2 indicate that the best rank for the proposed 
scheme is D = 4 (which will be used in the remaining experiments) 
and it is very close to the optimal MMSE. Studies with systems with 
different processing gains show that D is relatively invariant to the 
system size, which brings considerable computational savings to the 
proposed scheme and allows a very fast convergence performance. 
In practice, the rank D can be adapted in order to obtain fast conver- 
gence and ensure good steady state performance and tracking after 
convergence. 
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Fig. 3. BER performance versus number of received symbols. 



The BER convergence performance in a mobile communications 
situation is shown in Fig. 3. The channel coefficients are obtained 
with Clarkes model 1 18] and the adaptive filters of all methods are 
trained with 200 symbols and then switch to decision-directed mode. 
The results show that the proposed scheme has a much better perfor- 
mance than the existing approaches and is able to adequately track 
the desired signal. A complete convergence analysis of the proposed 
scheme, including tracking and steady-state performance, conditions 
and proofs are not included here due to lack of space and are intended 
for a future paper. 



6. CONCLUSIONS 

We proposed a novel reduced-rank scheme based on joint iterative 
optimization of filters with an implementation using RLS algorithms. 
In the proposed scheme, the full-rank adaptive filters are responsible 
for estimating the subspace projection rather than the desired signal, 
which is estimated by a small reduced-rank filter. The results for 
space-time interference suppression in a DS-CDMA system show a 
performance significantly better than existing schemes and close to 
the optimal MMSE in a dynamic and hostile environment. 
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